<html>
<head><meta charset="utf-8"><title>Profiling Cargo (again) · t-compiler/wg-parallel-rustc · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/index.html">t-compiler/wg-parallel-rustc</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html">Profiling Cargo (again)</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="179274801"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179274801" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179274801">(Oct 28 2019 at 20:55)</a>:</h4>
<p>Ok so I'm starting to collect some data and write some things to <a href="https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ" target="_blank" title="https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ">https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ</a></p>



<a name="179274856"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179274856" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179274856">(Oct 28 2019 at 20:56)</a>:</h4>
<p>I'm still in the process of doing so, but @cuviper I'm curious, do you know of good tools on Linux to profile contention?</p>



<a name="179275290"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179275290" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179275290">(Oct 28 2019 at 21:00)</a>:</h4>
<p>(that ping did not appear to work, <span class="user-mention" data-user-id="138448">@cuviper</span>)</p>



<a name="179275351"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179275351" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179275351">(Oct 28 2019 at 21:00)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> note that an alt build will have slower codegen (llvm asserts + IR verification)</p>



<a name="179275400"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179275400" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179275400">(Oct 28 2019 at 21:01)</a>:</h4>
<p>yep</p>



<a name="179275402"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179275402" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179275402">(Oct 28 2019 at 21:01)</a>:</h4>
<p>I'm just comparing it to itself</p>



<a name="179275409"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179275409" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179275409">(Oct 28 2019 at 21:01)</a>:</h4>
<p>not comparing it to nightly</p>



<a name="179275434"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179275434" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179275434">(Oct 28 2019 at 21:01)</a>:</h4>
<p>makes sense, just wanted to be sure :)</p>



<a name="179278976"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179278976" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179278976">(Oct 28 2019 at 21:42)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> what sort of profiling do you want? even those strace counts of <code>futex</code> are a pretty good indicator, which will be some balance of <code>FUTEX_WAIT</code> and <code>FUTEX_WAKE</code> calls</p>



<a name="179279072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279072">(Oct 28 2019 at 21:43)</a>:</h4>
<p>I suppose another way to put this is that a <code>Zthreads=28</code> build for me spends 870ms in <code>futex</code> syscall, but <code>-Zthreads=1</code> spends 46ms in there</p>



<a name="179279137"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279137" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279137">(Oct 28 2019 at 21:44)</a>:</h4>
<p>SystemTap has some locking examples, especially <code>mutex-contention.stp</code><br>
<a href="https://sourceware.org/systemtap/examples/keyword-index.html#LOCKING" target="_blank" title="https://sourceware.org/systemtap/examples/keyword-index.html#LOCKING">https://sourceware.org/systemtap/examples/keyword-index.html#LOCKING</a></p>



<a name="179279156"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279156" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279156">(Oct 28 2019 at 21:44)</a>:</h4>
<p>more worrisome though is <code>-Zthreads=1</code> spends 4us in <code>sched_yield</code> and <code>-Zthreads=28</code> spends 4 <em>seconds</em></p>



<a name="179279165"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279165" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279165">(Oct 28 2019 at 21:44)</a>:</h4>
<p>ok cool I might poke around at that</p>



<a name="179279206"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279206" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279206">(Oct 28 2019 at 21:45)</a>:</h4>
<p><code>sched_yield</code> is probably due to rayon threads spinning down</p>



<a name="179279218"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279218" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279218">(Oct 28 2019 at 21:45)</a>:</h4>
<p>niko's scheduling changes will hopefully help there</p>



<a name="179279339"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279339" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279339">(Oct 28 2019 at 21:46)</a>:</h4>
<p>(less spinning, more precise wakeups, etc.)</p>



<a name="179279410"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279410" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279410">(Oct 28 2019 at 21:47)</a>:</h4>
<p>for context you may want to see <a href="https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view#Crates-in-the-first-3-seconds" target="_blank" title="https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view#Crates-in-the-first-3-seconds">https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view#Crates-in-the-first-3-seconds</a> as well</p>



<a name="179279425"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279425" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279425">(Oct 28 2019 at 21:47)</a>:</h4>
<p>where basically I'm seeing that a huge amount of system time is spent in the first few seconds of the build</p>



<a name="179279432"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279432" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279432">(Oct 28 2019 at 21:47)</a>:</h4>
<p>so when there's tons of rustc parallelism then rustc internal parallelism is stressing the system quite a lot</p>



<a name="179279438"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279438" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279438">(Oct 28 2019 at 21:47)</a>:</h4>
<p>which if rayon has 28 threads (for mysefl locally) all spinning</p>



<a name="179279483"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279483" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279483">(Oct 28 2019 at 21:48)</a>:</h4>
<p>that's 28 * 28 threads all spinning</p>



<a name="179279488"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279488" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279488">(Oct 28 2019 at 21:48)</a>:</h4>
<p>which would explain that</p>



<a name="179279491"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279491" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279491">(Oct 28 2019 at 21:48)</a>:</h4>
<p>FWIW, I think those strace times are _wall_ times, not CPU usage</p>



<a name="179279504"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279504" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279504">(Oct 28 2019 at 21:48)</a>:</h4>
<p>oh this is <code>perf trace</code> if that makes a difference</p>



<a name="179279511"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279511" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279511">(Oct 28 2019 at 21:48)</a>:</h4>
<p>and I think it's wall time per syscall, right?</p>



<a name="179279536"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279536" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279536">(Oct 28 2019 at 21:48)</a>:</h4>
<p>or rather I don't think it can be overlal wall time</p>



<a name="179279568"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279568" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279568">(Oct 28 2019 at 21:49)</a>:</h4>
<p>a total of time from enter-&gt;exit the syscall</p>



<a name="179279573"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279573" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279573">(Oct 28 2019 at 21:49)</a>:</h4>
<p>b/c the whole build was 3 seconds and it say 4s was spent in <code>sched_yield</code></p>



<a name="179279577"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279577" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279577">(Oct 28 2019 at 21:49)</a>:</h4>
<p>yeah, summed up across all threds</p>



<a name="179279589"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279589" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279589">(Oct 28 2019 at 21:49)</a>:</h4>
<p>right, they can overlap</p>



<a name="179279727"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279727" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279727">(Oct 28 2019 at 21:51)</a>:</h4>
<p>I assume msec is milliseconds, no? so 4015660 is 4015.66 seconds</p>



<a name="179279731"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279731" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279731">(Oct 28 2019 at 21:51)</a>:</h4>
<p>lots of overlap, surely</p>



<a name="179279877"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279877" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279877">(Oct 28 2019 at 21:53)</a>:</h4>
<p>so I thought so, but I think "msec" is "microsecond"</p>



<a name="179279881"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279881" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279881">(Oct 28 2019 at 21:53)</a>:</h4>
<p>I find <code>perf</code> output pretty bad from time to time...</p>



<a name="179279900"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279900" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279900">(Oct 28 2019 at 21:53)</a>:</h4>
<p>I definitely didn't wait for an hour</p>



<a name="179279913"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279913" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279913">(Oct 28 2019 at 21:53)</a>:</h4>
<p>so from this I see two things:</p>



<a name="179279966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279966">(Oct 28 2019 at 21:54)</a>:</h4>
<ul>
<li>Niko's scheduler changes will be massively beneficial to avoid idle threads spinning</li>
</ul>



<a name="179279968"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279968" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279968">(Oct 28 2019 at 21:54)</a>:</h4>
<p>no, but spread across 3998 <code>clone</code> tasks (process and/or threads)</p>



<a name="179279974"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279974" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279974">(Oct 28 2019 at 21:54)</a>:</h4>
<ul>
<li>I think we still need to investigate lazily spinning up threads</li>
</ul>



<a name="179279997"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179279997" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179279997">(Oct 28 2019 at 21:54)</a>:</h4>
<p>I wish I could drill more into why <code>futex</code> is taking so much longer</p>



<a name="179280007"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280007" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280007">(Oct 28 2019 at 21:54)</a>:</h4>
<p>but that seems like it's possibly just bland everyday contention which would be harder to optimize</p>



<a name="179280025"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280025" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280025">(Oct 28 2019 at 21:55)</a>:</h4>
<p>but <code>jobserver</code> at least seems entirely unrelated to the slowness that I'm seeing</p>



<a name="179280036"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280036" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280036">(Oct 28 2019 at 21:55)</a>:</h4>
<p>or rather, it's hidden by the massive costs of <code>sched_yield</code></p>



<a name="179280119"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280119" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280119">(Oct 28 2019 at 21:56)</a>:</h4>
<p>there are about 2x read/write syscalls, and that is likely related to the jobserver</p>



<a name="179280125"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280125" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280125">(Oct 28 2019 at 21:56)</a>:</h4>
<p>but the main costs seem to be elsewhere</p>



<a name="179280154"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280154" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280154">(Oct 28 2019 at 21:57)</a>:</h4>
<p>can you cargo-patch to niko's rayon branch to see its effects?</p>



<a name="179280182"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280182" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> cuviper <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280182">(Oct 28 2019 at 21:57)</a>:</h4>
<p>er, well, I guess you need rustc-rayon with those changes, not as simple</p>



<a name="179280198"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280198" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280198">(Oct 28 2019 at 21:57)</a>:</h4>
<p>yeah :(</p>



<a name="179280208"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179280208" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179280208">(Oct 28 2019 at 21:57)</a>:</h4>
<p>if we could get that up to a try build though that'd be great</p>



<a name="179286482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179286482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179286482">(Oct 28 2019 at 23:27)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> here is a try build with no jobserver support (<code>efc4e1038fd187f9817854c353e56a74d6f49709</code>, see <a href="https://github.com/rust-lang/rust/pull/65908" target="_blank" title="https://github.com/rust-lang/rust/pull/65908">https://github.com/rust-lang/rust/pull/65908</a>) which also disables llvm asserts etc so alt and non-alt are fine to compare parallel vs. non-parallel w/o jobserver. I've also kicked off a perf.rlo run to confirm that jobserver is not harming the current case</p>
<p>it should be compared against <a href="https://github.com/rust-lang/rust/commit/8d78bf6b273848d17da8f5c92162c6a6b9b10dfd" target="_blank" title="https://github.com/rust-lang/rust/commit/8d78bf6b273848d17da8f5c92162c6a6b9b10dfd">8d78bf6b273848d17da8f5c92162c6a6b9b10dfd</a> if you wan</p>



<a name="179293687"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179293687" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179293687">(Oct 29 2019 at 02:02)</a>:</h4>
<p>er, if  you want to compare against "real" master</p>



<a name="179293688"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179293688" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179293688">(Oct 29 2019 at 02:02)</a>:</h4>
<p>i.e., without jobserver changes</p>



<a name="179336313"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179336313" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179336313">(Oct 29 2019 at 14:52)</a>:</h4>
<p>Thanks <span class="user-mention" data-user-id="116122">@simulacrum</span>! Updated the doc - <a href="https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view" target="_blank" title="https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view">https://hackmd.io/oUdvUU2lTk2ZxfOWQBT9xQ?view</a> - with that information</p>



<a name="179336351"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179336351" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179336351">(Oct 29 2019 at 14:52)</a>:</h4>
<p>my conclusion is that removing the jobserver does help, but not as much as you might think</p>



<a name="179336456"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179336456" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179336456">(Oct 29 2019 at 14:53)</a>:</h4>
<p>I think that we still have the same two conclusions to start with and get a lion's share of improvements</p>



<a name="179336466"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179336466" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179336466">(Oct 29 2019 at 14:53)</a>:</h4>
<p>which is to test out the new scheduler which is more eager to shut threads down</p>



<a name="179336519"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179336519" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179336519">(Oct 29 2019 at 14:54)</a>:</h4>
<p>and additionally a separate change to spin up threads lazily instead of all at once</p>



<a name="179336676"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling%20Cargo%20%28again%29/near/179336676" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Profiling.20Cargo.20(again).html#179336676">(Oct 29 2019 at 14:55)</a>:</h4>
<p>For those following this, I think that's all the investigation I think needs to be done before next week's mtg, I'm pretty confident that this is enough data to dig in and start a round of improvements. If others though think there's something else that should be investigated, let me know!</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>